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(g) Multistage growable optical packet switching arrangement with bypass. 

(57) A growable packet switching arrangement 
where the distribution network (23) blocking 
probability is substantially reduced because the 
networic has both switch links and chute links, 
and the networtc nodes (e.g. 30) include both a 
switching means (40) interconnecting succes- 
sive stage switch links and a plurality of non- 
switching, chute connection means (50) inter- 
connecting successive stage chute links. A net- 
work node can transfer a packet, being received 
on a switch link, to any selected one of the 
chute connection means of that node for trans- 
mission on a chute link. The network nodes are 
relatively simple and inexpensive because they 
store only the first few bits needed to route an 
ATM cell. The blocking probability is further 
reduced when the number of chutes per node is 
increased. The number of chutes may be based, 
for example, on the number of switch link in- 
puts per node. 
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Technical Field 

This Invention relates to connmunication systenris. 
Background and Problem 

The term broadband covers a host of new products, technologies, services, and networks. One way to 
define broadband networks Is to categorize them as those networks that support services requiring bit rates 
well above one megabits per second. Business and residential subscribers will be connected to broadband net- 
works via a common access, operating at 150 megabits per second or above, that can handle a range of dif- 
ferent broadband service types. ATM (asynchronous transfer mode) has been chosen as the communication 
principle on which broadband networks will be based. Afutura broadband ISDN (integrated servtees digital net- 
work) will offer the flexibility needed to handle diverse services ranging from basic telephone service to high 
speed data transfer, videotelephony, and high quality television distribution. The key to this flexibility is ATM 
which carries digital information in special cells. This allows the network to be used efficiently by applications 
and services with widely differing bandwidth requirements and call characteristics. 

Agrowabie packet switching arrangement disclosed in a paper by K. Y. Eng et al., entitled The Growable 
Switch Architecture: A Self-Routing Implementation for Large ATM Applications", international Conference on 
Communications, June 1991, has been proposed for use in ATM applications. In the disclosed arrangement, 
a partition is made between a front-end distribution network and a column of output packet switching modules. 
The outputs are divided into groups of n lines each. All incoming cells are routed through the front-end distrib- 
ution network for substantially instantaneous delivery based on their destination output group addresses. As 
such, the distribution network must not buffer full cells and must instead perform its routing function for all the 
cells arriving in each packet time slot. Since the outputs are grouped into n lines each, there are a total of N/n 
output groups. For each output group, the corresponding output packet switching module has m (m>=n) Inputs, 
meaning that up to m cells can be accepted for that output group in each time slot. The output packet switching 
module has buffers for storing full received cells. In an Lx N packet switch, up to L cells can arrive simultane- 
ously for a particular output group so the design range of m Is n<=m<=L. According to the generalized knockout 
principle, it is possible to pick m«N to yield an arbitrarily small cell loss probability that is acceptable for the 
switching system design, even under the most general traffic assumptions. The maximum number of cells ac- 
cepted into each output group is restricted to m; If there are more than m cells destined to the same output 
group in a time slot, the excessive cells are simply dropped. Although the general characteristics of a growable 
packet switching arrangement are known, the design of such an arrangement with a low cost, high perfor- 
mance distribution network remains an important problem. 

A high performance, self-routing ATM switch, referred to as a rerouting network (FIG. 39) is disclosed in 
a paper by S. Urushidani entitled, "Rerouting Network: AH igh-Performance Self-Routing Switch for B-ISDN", 
IEEE Journal on Selected Areas in Communications. Vol. 9, No. 8. October 1 991. Although the Urushidani 
paper contains no disclosure, teaching or suggestion concerning growable packet switching arrangements, the 
performance of the disclosed ATM switch Is enhanced by a "rerouting" algorithm applied to a particular mul- 
tistage interconnection algorithm. The interconnection algorithm offers many access points to the output and 
resolves output contention by including buffers at each switching stage. The rerouting network Is a multistage 
interconnection network with more than log2L switching stages. The stages are interconnected by routing links 
and bypass links. The algorithm Is applied to embed plural banyan networks within the switch. That is, the 
partial networks composed of switching elements and routing links from switching stages 1 to 4, from 2 to 5, 
and so on are banyan networks, because a single path always exists between all input-output pairs. Therefore, 
a cell that fells to take its desired route due to cell contention at a given stage can restart its routing from the 
next stage through a banyan network formed in the following ioQjL stages. This restart is referred to as "re- 
routing". On the other hand, the bypass links route the cell, having reached its destination, to the output line 
without switching at each switching element. Although the rerouting network has good performance— for ex- 
ample, relatively low blocking-it is not suitable for use as the distribution networkof a growable packet switching 
arrangement One of the key properties of a growable packet switching arrangement Is that the nodes of the 
distribution network should not be required to buffer full packets. The rerouting network nodes each include 
sufficient buffers to store full 53-byte ATM cells for later transmission. Accordingly, the nodes are complex and 
expensive in sharp distinction to the distribution network nodes needed in a growable packet switching ar- 
rangement. 

In view of the foregoing, a need exists in the art for a growable packet switching arrangement with a low 
cost, high performance distribution network. 
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Solution 

This need is met and a technical advance is achieved In accordance with the principles of the invention 
In an exemplary growabie packet switching arrangement where the distribution network blocking probability 

5 is substantially reduced because the network has both switch links and chute links, roughly comparable to the 
rerouting network routing links and bypass links, respectively, and the network nodes include both a switching 
means interconnecting successive stage switch links and a plurality of non-switching, chute connection means 
interconnecting successive stage chute links. A network node can transfera packet, being received on a switch 
link, to any selected one of the chute connection means of that node for transmission on a chute link. The net- 
to work nodes are relatively simple and inexpensive because they store only the first few bits needed to route 
an ATM cell, rather than storing the full 53 bytes for transmission at a later time as In the rerouting network 
nodes. The blocking probability is further reduced when the number of chutes per node is increased. The nunrv 
ber of chutes may be based, for example, on the number of switch link inputs per node. 

A packet switching arrangement in accordance with the invention has L input ports, M intermediate ports 

IS and N output ports, where L, M and N are positive Integers greater than two. The arrangement Includes a dis- 
tribution network and N/n output packet switching modules, where n and N/n are positive integers with N>n>1. 
The distributton network switches packets from the L input ports to the M intermediate ports without storing 
full received packets. The packet switching modules each comprise means for switching packets from a distinct 
group of m intermediate ports to a distinct group of n output ports, where m is a positive integer with m>n. 

20 Each packet switching module further comprises means for storing full received packets. The distribution net- 
work comprises S stages of nodes. Each node of the first {S-1) node stages comprises an X x Y switching 
means, where S>3, X>1, and Y>1. The distribution network further comprises (S-1) link stages interconnecting 
successive node stages with each link stage comprising chute links and switch links. The X x Y switching means 
of each node of the interior (S-2) node stages receives packets on X switch links and selectively transmits pack- 

25 ets on Y switch links. Each such node includes a plurality of chute connection means each connected to only 
one chute link of the preceding link stage and to only one chute link of the following link stage for transmitting 
a packet, being received from the preceding stage chute link, onto the following stage chute link. Each such 
node also Includes means for transferring a packet, being received on a switch link by the X x Y switching 
means, to any selected one of the chute connection means for transmission on the following stage chute link. 

30 Illustratively, X=2, Ys2 and the number of chute connection means of each node of the Interior (S-2) node 

stages is Xm/n=4. Ail four of the chute connection means are connected via chute links to only one node of 
the following node stage. The selected chute connection means is selected based on the absence of a packet 
on the preceding stage chute link. A packet is transferred from the 2 x 2 switching means to a chute connection 
means based on destination information included in the packet. The transfer is made unless packets are pres- 

35 ent on the four preceding stage chute links connected to the four chute connection means. 

The 2x2 switching means of each node of the first node stage receives packets from two input ports and 
selectively transmits packets on two switch links. The first stage node Includes four chute connection means 
each connected to only one chute link of the following link stage for transmitting a packet thereon. The first 
stage node also includes means for transferring a packet, being received from one of the two input ports, to 

40 any selected one of the four chute connection means for transmission on the following stage chute link. 

Each node of the last node stage includes means for receiving packets on two switch links, and four chute 
connection means each connected to only one chute link of the preceding link stage and to only one of the M 
Intermediate ports, for transmitting a packet, being received from the preceding stage chute link, to the one 
Intermediate port. The last stage node also includes means for transferring a packet, being received on a switch 

45 link by the receiving means, to any selected one of the four chute connection means for transmission to the 
one intermediate port. The receiving means discards a packet being received on a switch link but not being 
transferred to a chute connection means. No packets are discarded by any node of the first (S-1) node stages. 

The distribution network includes a number of partitions each associated with a distinct output packet 
switching module. Each partion includes all of the network chute links leading to the associated output packet 

so switching module and all of the network nodes interconnected by those chute links. There are n/X=2 nodes 
per node stage in a partition. The (S-1) links stages include global link stages having inter- partition switch links 
and local link stages having no inter-partition switch links. The network nodes include global nodes preceding 
global link stages and local nodes preceding local link stages. The global nodes route packets in accordance 
with a first routing method; the local nodes route packets in accordance with a second routing method. 

55 

Drawing Description 

FIG. 1 is a diagram of an exemplary growabie packet switching arrangement Including a packet distribution 

3 



EP0 590 865 A2 



network; 

FIG. 2 is a diagram of two optical hardware modules included in the arrangement of FIG. 1; 
FIGS. 3-4 are diagrams of optical hardware used in a prior art network; 

FIG. 5 is a diagram showing the light beams generated by binary phase gratings such as those shown in 
5 FIG. 2; 

FIG. 6 is a diagram of the topology of the network of FIG. 1; 

FIG. 7 is a diagram of the functional elements of one node in the network as shown In FIG. 6; 

FIGS. 8-11 are flow charts of the contention resolution method used in global nodes of the network of 

FIG. 6: 

10 FIGS. 12-15 are flow charts of the contention resolution method used in local nodes of the network of 

FIG. 6; 

FIG. 16 is a diagram of the network of FIG. 6 illustrating parameters used in modeling network perfor- 
mance; 

FIGS. 17-26 are diagrams showing the performance of networks of the type shown In FIG. 6 but having 
15 varying network parameters; 

FIGS. 27-36 are diagrams showing the various elements of the node of FIG. 7 in greater detail; 
FIG. 37 is a diagram of the prior art, growable packet switching architecture; 
FIG. 38 Is a diagram illustrating the relation of certain parameters of the architecture of FIG. 37; and 
FIG. 39 is a diagram of a prior art network. 

20 

General Description 

One specific illustrative embodiment of a growable, packet switching arrangement 10 is shown in FIG. 1. 
Arrangement 10 includes a packet distributk)n network 23 implemented using free-space optics. Packets are 

25 received on 16 Inputs by a synchronizer 21 which aligns incoming packets for transmission in packet time slots. 
The aligned packets are converted to optical signals by an E/0 converter 22 and transmitted by a 4 x 4 fiber 
cable array 31 to packet distribution network 23. Network 23 packet switches each incoming packet for trans- 
mission on one fiber of an 8 x 4 fiber cable array 32. Network 23 includes a lens 33, seven optical hardware 
modules OHM0-OHM6, and a lens 34. The optical signals transmitted from network 23 are converted back to 

30 electrical signals by an O/E converter 24 for transmission to four output packet modules 25-28. The network 
23 topology is shown in FIG. 6. 

Network 23 includes seven planar arrays D0-D6 of FET-SEED devices implemented using the technology 
disclosed in a paper by T. K. Woodward et al., entitled "Operation of a Fully Integrated GaAs-AI^Gai. xAs FET- 
SEED: A Basic Optically Addressed Integrated Circuir. IEEE Photonics Technoloqv Letters. Vol. 4, No. 6, June 

35 1 992. The optical hardware required for two consecutive optical hardware modules OHM2 and OHM3 is shown 
schematically in FIG. 2. The linearly polarized light emitted from the laser diode power supply 81 is collimated 
by the collimation lens 82. The collimated beam Is then routed through a pair of Rtsley prisms 83. 84, which 
can be rotated to point the beam to its appropriate location on the device array. This pointed beam is then 
passed through a spot array grating 85 to create an array of clock beams that can be used to interrogate the 

40 state of each of the windows in the device array D2. This array of beams is then passed through a quarter 
wave-plate retarder 86 to convert the light from a linear polarization state to a circular polarization state. The 
array of dock beams is then imaged by lens 87 through the transparent spacings on patterned mirror reflector 
88 before being re-collimated by a third lens 89 and re-converted into a linear polarization state by quarter 
wave-plate retarder 90. This linear polarized light passes straight through polarization-sensitive beam splitter 

45 91 before being routed through a quarter wave-plate retarder 92, converting it Into circularly polarized light. 
The light is then Imaged by an objective lens 93, creating dock spots in the windows of device array D2. Each 
of these dock spots is modulated (absorbed or reflected) by the device window, and the reflected light is routed 
back through objective lens 93 which now re-collimates the light. This re-coHimated light is then passed through 
quarter wave-plate retarder 92 which converts it back into linear polarized light, but its plane of polarization is 

50 now oriented so that the light is reflected to the right by polarization-sensitive beam splitter 91. The light is 
then routed through another set of Risiey prisms 94, 95, which can be used to point the beams into the windows 
of the second device array D3. The pointed beams then pass through a 1 x 3 binary phase grating 96, which 
provides a Banyan interconnection pattern. The output beams from grating 96 are directed into a second po- 
larization-sensitive beam splitter 191 and reflected up. These beams pass through quarter wave-plate retarder 

55 190, which converts them into the circular polarization state, and then through lens 189. They are then im- 
aged onto the reflective mirrors on patterned mirror reflector 188 and reflected back down through lens 189 
and then through quarter wave-plate retarder 190 which converts them back into a linear polarization state. 
These beams then pass through polarizatton-senslth^e beanrvsplitter 191 and through quarter wave-plate re- 
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tarder 192 toward the second objective lens 193. The nnodulated signal beams from the first device array D2 
are then imaged as spots within the windows of the second device array D3, setting the digitai states of those 
devices. Note that the only difference between OHM2 and OHMS is that the period of grating 196 is half that 
of grating 96. This results in the different interconnection patterns in network 23 (FIG. 6). The INIT, LOAD and 
5 CLK signals are described later herein. 

Detailed Description 

Background on the Growable Packet Switch Architecture 

10 

The general growable packet switch architecture contains two basic sub-systems: the distribution network 
and output packet modules (FIG. 37). These two sub-systems perform very distinct functions. The distribution 
network routes each packet to the output packet module that Is connected to the packet's desired output port. 
Thus, if a packet is destined for output port #0, then the distribution network merely needs to route the packet 

iS to any one of the m input lines connected to output packet module #0, assuming that one of the m input lines 
is idle. The distribution network supports L input ports, and it also supports M=Nm/n intermediate ports (or 
distribution network output ports) that are routed to the mxn output packet modules, where L, N, and n are all 
a power of 2. Each of the output packet modules has access to n output ports, so a total of K=N/n output packet 
modules are required. Each output packet module stores the arriving packets In FIFO queues, and It then routes 

20 the packets at the front of each of the m FIFO queues to their desired output ports. Thus, In addition to providing 
the FIFO queues, the output packet module also provides the functionality of a small switching network. The 
overflow problems associated with the finite-length queues within the output packet module can be analyzed 
to determine satis^ctory queue lengths based on the traffic load. 

An interesting feature of the growable packet switch Is the manner in which it provides arbitration and con- 

25 tentton resolution between multiple packets that are destined for the same output port. Arbitration determines 
which of the input packets is given priority when contention exists, and those packets which are not able to be 
instantly routed to their desired output are buffered at FIFO queues in the output packet modules so that they 
can then be transmitted by the output packet module to their desired output port at a later time. The Knockout 
Principle is the contention resolution scheme used in the growable packet switch, and it takes advantage of 

30 the statistical nature of packet traffic to determine the number of queues that must be associated with each 
output port to yield desired operating characteristics. According to the generalized Knockout Principle, if the 
arriving packets on diffarent input lines are statistically independent, then groups of n output ports can be as- 
sociated with a single output packet module containing m input lines and m FIFO queues, and the probability 
that more than m packets will be simultaneously destined for the n output ports can be engineered to be very 

35 low by selecting an appropriate value of m. (The basic knockout principle is disclosed In a paper by Y. S. Yeh 
et al., entitled The Knockout Switch: A Simple, Modular Architecture for High-Performance Packet Switching", 
ISS 87. AT&T Technical Papers . If more than m packets are simultaneously destined for the same output packet 
module, then only m of the packets are accepted (loaded Into the m FIFO queues), and the rest of the packets 
are dropped. Higher level protocols can be used to initiate the re-transmission of dropped packets. 

40 If the network size (LxN) is large and if pi. is the network loading (the probability that a packet arrives on 

an input line during a particular packet slot), it can be shown that the probability that a packet is lost within 
the growable packet switch is approximately given by: 

^ PCpackct loss)=(l-m/(npL)]ll- £ (npL) V"''Vk!]+(npL)'"c""^Vm! [1] 

For example, if the switch is 90 percent loaded (pl=0.9) and if the output packet module has n=8 output ports 
and m=32 FIFO queues, then Eq. 1 indicates that a network with large L and N would experience a packet 

50 loss probability of approximately 4x1 0-^^ Eq. 1 also indicates that larger values of n require smaller m/n ratios 
to produce a particular packet loss prok>ability. However, larger values of n also require more complicated func^ 
tionality (and cost) within the output packet modules. If the minimum acceptable packet loss probability is de- 
fined (somewhat arbitrarily) to be then the required values of m/n are illustrated in FIG. 38 as a function 
of n. For most applications, it is sufficient that the exemplary growable packet switching arrangements herein 

55 have a probability of packet loss less than the square root of the expression of Eq. 1 . 
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interconnections Based on Free-Space Digital Optics 

The interconnections between devices in the growable packet switch can be provided using free-space 
digital optics. Avery simple arrangement of optical components that can provide many useful interconnection 

5 patterns with low power loss is shown in FIG. 3. This arrangement is based on a simple 2-lens, infinite conju- 
gate, telocentric imaging system set up to provide 4f imaging. However, a binary phase grating (diffiractive ho- 
logram) has been added at the pupil plane between the two Imaging lenses to provide the necessary beam- 
steering operations. Changes in the periodic structure of the grating can diffract different amounts of optical 
power from the original beam into different spots in the output image plane. The grating in FIG. 4 routes each 

10 input beam to three spots in the output image plane. This arrangement of hardware (FIG. 4) was proposed for 
use in a multistage interconnection network known as the Trimmed Inverse Augmented Data Manipulator 
(TIADM) network disclosed in a paper by T. J. Cioonan et al., entitled "Optical Implementation and Performance 
of One-Dimensional and Two-Dimensional Trimmed Inverse Augmented Data Manipulator Networks for Mul- 
tiprocessor Computer Systems", Optical Engineering . April 1989, Vol. 28, No. 4, pp. 305-314. If some of the 

IS beams in the TIADM network (iight-colored links in FIG. 5) are blocked, then the connections for the Banyan 
network are provided by the remaining dark-colored links. A2D-to-3D conversion of this Interconnection pat- 
tern is performed to make it usable with planar device arrays. This conversion is described in detail within the 
literature. 

The techniques used to label the link-stages, the node-stages, and the nodes within a node-stage are 11- 
20 lustrated In FIG. 5. If the network contains S node-stages, then these node-stages are labeled from 0 to S-1, 
and the link-stage following node-stage i is labeled link-stage i. FIG. 5 also shows the fixed link-stage connec- 
tions required between adjacent node-stages. If there are L/2=N/2=Q==2<i nodes in each node-stage labeled 0 
to Q-1, then the Banyan interconnection within link-stage i must provide a connectbn from every node (U) in 
node-stage i to two nodes (Zo and Zi) in node-stage i+L The Banyan can be described by two node-mapping 
25 functions {B{* and B|^) on the address of node U, where each mapping function describes the connection pro- 
vided by one of the two output links from node U In node-stage i to one of the two nodes Zo and Zi in node- 
stage 1+1. In the equatbn below, the node address U is represented by a q-bit binary representation (Uq.Uq. i, 
• • .U2,Ui), and the node address Ze is represented by a q-bit binary, representation (Zq.Zq- 1,— ,Z2.Zi). The map- 
ping from node U to node Ze can therefore be described by the general expression: 

30 (Zq.Zq . 1, ,Z2,Zi) = B,«[(Uq,Uq . i. " .Uq . , + i,Uq - |,Uq . , . i. -.Uz.Ui)] = (Uq.Uq . i, •", Uq . , + 1,9 XOR Uq . ,,Uq . 

I - 1. • •.U2,Ui). [2] 

where 0 is the output link identifier (B^O for the straight output link, and 0=1 for the angled output link). One of 
the two mappings described by Eq. 2 implements the straight connection to the node directiy in line with the 
source node, while the other mapping implements the angled connection to a node that is 29->- 1 node positions 
35 away fit>m the source node. 

The Chuted-Banyan (Canyan) Distribution Network 

Because the Canyan (Chuted-Banyan) distribution network 23 Is used within a growable packet switching 
40 arrangement 1 0, it is implemented as a synchronous network with all of the arriving fixed-length packets buf- 
fered and synchronized at the input of the network, if the incoming packets are 53-byte ATM packets, the virtual 
path indicators (VPIs) used for transmission equipment (or the virtual channel indicators (VCIs) used for switch- 
ing equipment) are extracted from the 5-byte packet headers and routed through a translation table at the front- 
end of the network. The output of the translation table contains a new VPI (VCI) that is over-written on top of 
45 the old VPI (VCI). It also contains the actual destination address of the output port to which the packet is des- 
tined, and this information is prepended to the 53-byte ATM packet As a result of this prepended address, a 
slight data rate speed-up Is required between the incoming data lines and the links within the Canyan distrlt>- 
ution network. 

The network structure of Canyan distribution network 23 is shown in FIG. 6 within growable packet switch- 
so ing arrangement 10 having L=16 input ports, N=16 output ports, and output packet modules 25-28 with m=8 
inputs (FIFO queues) and n=4 output ports. Thus, growable packet switching arrangement 10 provides 
K=N/na4 output packet modules. Canyan distribution network 23 supports L=16 input ports and M=Nm/n=32 
intermediate ports (or network 23 output ports); network 23 provides an effecth/e fenout F=m/n=2. Canyan 
distribution network 23 contains S=7 node-stages. The number, S, of distribution network stages is reduced 
55 by making (S+1), the number of stages including the stage of output packet modules, less than [iog2(max 
(L,N))p/2, the minimum number of stages in a multi-stage sorting networkof 2x 2 switching means, K. E. Batch- 
er, "Sorting Networks and Their Applications." The Proceedings of AFIPS . 1988. (S+1) is also made greater 
than log2N so that complete connectivity is achieved between input ports and output ports. For the remainder 
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of this description, it is assumed that the number of input ports (L) of packet switching arrangement 10 is equal 
to the number of output ports (N) of arrangement 10. 

Canyan distribution networic 23 can be viewed as two distinct sub-networlcs (the switching sub-network 
and the chute-multiplexing sut>-network) overlaid on top of one another. As a result, the nodes In FIG. 6, (e.g., 

5 30) are effectively split into two distinct parts: the switching means 40 (the portion of each node below the 
dashed line) and the chute-multiplexing portton 50 (the portion of each node above the dashed line). The 
switching means of each node provides the functionality required for the routing of each packet to an appro- 
priate output chute in the chute-multiplexing sub-network, while the chute-multiplexing portk)n of each node 
comprises four chute connection means which merely transport the packets toward the FIFO queues in the 

10 output packet modules 25-28. Because of similarities between the beanrvsteering operations required for their 
link-stage interconnections, both the switching sub-network and the chute-multiplexing sub-network share the 
same imaging optics to provide the connections between adjacent node-stages. 

The Switching Sub»Network 

IS 

In the Canyan network, the switching sub-network is a standard Banyan network containing 2x2 self- 
routing switching means (the lower portion of the nodes in FIGS. 6 and 7). Two packets arriving at the switching 
means input ports can be routed on straight paths or on crossed paths to the two switching means output ports. 
The links connecting the outputs from the switching means of the nodes in one node-stage to the switching 

20 means of the nodes in the next node-stage are referred to herein as switch links and provide a Banyan Inter- 
connection pattern (Eq. 2). The Banyan pattern in an L=N-input Canyan network is repeated after every set 
of log2(N)-1 link-stages, because fiili connectivity is provided after this number of link-stages. Thus, link-stage 
f will provide the Banyan connecttons defined by the mapping functions B^ftnodpog^N)- 1] and BiftnodoaQ2(N)- ijf where 
A mod B Is defined as the integer remainder of the quotient A/B and the mapping functions are defined in Eq. 

2^ 2. To identify the link-stage type within a Canyan network, a network parameter known as the connection va- 
riable (V) is defined. In any link-stage i of an N-input Canyan network, the connectton variable for that particular 
link-stage is defined to be: 

V = log2(N)-1-imodDog2(N)-1]. [3] 
The connectk>n variables for each of the link-stages in the Canyan network of FIG. 6 are displayed at the top 
^ of the figure. 

If a particular packet is destined for an output port emanating from output packet module j, then the goal 
of the switching sub-network is to route that packet on the Banyan connections to any node in any node-stage 
that is in any horizontal row connected via the straight output chutes to output packet module j. Once the packet 
has arrived at any of these desired nodes, then the packet is transferred within the node to the chute- 
^ multiplexing portion of the node (the upper portion of the nodes in FIGS. 6 and 7). 

The Chute«Multlplexing SulnNetwork 

The chute-multiplexing sub-network is a simple network with Nnn/n straight output links, referred to herein 
^ as chute links, interconnecting the chute-multiplexing portion of nodes (the upper portion of the nodes in FIGS. 
6 and 7). If X is the number of switching means inputs per node (In this example, X=2), the Xm/n output chutes 
leaving the chute-multiplexing portion of node U in node-stage 1 are connected directly to the Xm/n inputs on 
the chute-multiplexing portion of node U in node-stage \*^, and the Xm/n output chutes leaving the chute- 
multiplexing portion of node U in the last node-stage are connected via fiber optic bundles and an O/E converter 
^ 24 directly to FIFO queues in the associated output packet module. The chute-multiplexing portion of node U 
comprises Xm/n chute connection means. All of the output chutes require straight link-stage connections, which 
are provided by the Banyan interconnection optics of FIG. 4. The chute-multiplexing sut>-network permits pack- 
ets that have arrived at a node in their desired output row to be multiplexed onto the first available (Idle) output 
chute of the 2m/n output chutes emanating from the node. Once a packet is routed to an idle output chute, it 
^ is transported straight across that output chute from node to node until it arrives at the FIFO queue in the output 
packet module. In effect, a packet that Is routed to an output chute takes possesston of that chute for the entire 
packet slot, and no other packets are permitted to use that chute during the entire packet slot. 

A Routing Example 

An example of a packet being routed through Canyan distribution network 23 is shown by bold, solid links 
in FIG. 6. This packet is to be routed from input port 11 to output port 5 within growable packet switching ar- 
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rangement 10. Since output port 5 is attached to output packet module 26. the packet can be routed down any 
one of the eight output chutes directed at output packet module 26. These output chutes are numbered 8-15. 
In link-stage 0 and link-stage 1, the packet passes through links in the switching sut>-network until it arrives 
at node 3 in node-stage 2. Since node 3 has output chutes that are connected to output packet module 23. 

5 the packet can be transferred to the chute-multiplexing sub-network and directed to output chute 12. It then 
passes straight through the remaining stages on output chute 12, ultimately arriving at one of the FIFO queues 
In output packet module 26. Output packet module 26 then routes the packet to its destination (output port 5). 

Another path connecting input 11 to output 5 is shown by the bold, dashed links in FIG. 6. This alternate 
path would be used by the packet if it were deflected from its desired path in node-stage 0. This deflection 

10 could occur as a result of another packet propagating through the network that is already using the desired 
path. Once a packet is deflected, it propagates through a minimum of log2(N/n)-i-1 link-stages before it can be 
transferred to the chute-multiplexing logic However, if the Canyan network is designed with a sufficient num- 
ber of link-stages, then the alternate paths will typically permit most of the deflected packets to be routed to 
their desired destinations. 

IS 

Nodes In the Canyon Distribution Network 

To provide the routing functions described in the previous section, the 2x2 switching means of each node 
within the Canyan distribution network performs five basic tesks. These tasks are: 1) recognition of activity 

20 on an input link, 2) calculation of a packet's desired output link, 3) recognition that a packet can be transferred 
from the switching sub- network to the chute-multiplexing sub- network, 4) arbitration of packet contention prob- 
lems, and 5) routing of the packets to appropriate output links or to the chute-multiplexing portion of the node. 
The chute-multiplexing portion of each node also performs several tasks, including 1) recognition of activity 
on an input chute, 2) passing of packets from input chutes to output chutes. 3) multiplexing of a new packet 

25 from the switching means of the node onto the first available output chute, and 4) informing the switching 
means of the node of the availability (or lack of availability) of idle output chutes. 

The basic hardware within a single node 30 (FIG. 6) is illustrated in FIG. 7. In switching means 40, two 
link Inputs (labeled input A and input B) and two output links (labeled output C and output D) are provided in 
the switching means of the node. Input link A is assumed to come from the straight link within the Banyan in- 

30 terconnection, and input link B is assumed to come from the angled link (angled either up or down) within the 
Banyan interconnection. Output link C is assumed to go to the straight link within the Banyan interconnection, 
and output link D Is assumed to go to the angled link (angled either up or down) within the Banyan intercon- 
nection. Log2(N/n)i-1 of the header bite (described below) that arrive on an input link are buffered in a 
log2(N/n)+4 bit shift register 41 , 42, and the buffered bits are routed to the path hunt logic 43 which determines 

35 appropriate output links for each of the packets. (Note: The three extra bits in the shift registers 41 , 42 provide 
sufficient delay for the operations of the path hunt logic 43 to be completed before the outputs of the shift 
registers 41, 42 are sent to output multiplexers 44, 45. Output multiplexers 44, 45 route the packete to their 
appropriate output links based on the select signals produced by path hunt logic 43. To calculate appropriate 
output links for the packete, the path hunt logic is supplied with several pieces of Information, including the 

40 destination addresses and the activity bite for the packets on inputs A and B, the node number (to identify when 
a packet is in its desired row), the connection variable V (to identify the pivot bit used for routing), and a load 
signal (to indicate when the packet header is appropriately latched in the input buffer so that path hunt proc- 
essing can begin). The node number and the connection variable can be hard-wired values that would change 
for each node. 

45 Path hunt logic 43 also determines if any of the arriving packets should be transferred to chute-multiplexing 

portion 50 of the node. To calculate whether a packet should be transferred to an output chute, path hunt logic 
43 is supplied with several pieces of information, including the destination addresses and the activity bits for 
the packets on inpute Aand B, the node number, the connection variable (V), and a flag indicating if all of the 
chutes are full, (the flag being generated by first idle chute locator 54 activity bit). If a packet is transferred 

50 into chute-multiplexing portion 50 of the node, path hunt logic 43 generates the appropriate select signals for 
a transfer multiplexer 46, which routes packet A or packet B up into chute-multiplexing portion 50. Path hunt 
logic 43 also sends a chute transfer signal to output chute selector 55 so that the multiplexer 56, 57 associated 
with the first available (idle) output chute can be enabled to receive the transferred packet. First idle chute 
locator 54 receives a reset pulse to initialize the logic at the beginning of each packet interval. Chute- 

55 multiplexing portion 50 of the node also has logic to pass the packete on the 2m/n incoming chutes to the 2nn/n 
output chutes. Only one flip-flop delay is required for re-timing of bite as they pass through chute- multiplexing 
portion 50 of the node in each node-stege. However, an implementation using only one flip-flop resulte in dif- 
ferent network delays for packete that have entered the chutes in different node-stages, so the logic associated 
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with the FIFO queues of the output packet modules must be modified to identify the start of a packet (by lo- 
cating the activity bit) and toad the packet into the FIFO queues as it arrives. Another approach that eliminates 
this variable delay would add lo92(N/n)+4 bit shift registers to each of the chute Input ports, but this scheme 
increases the node hardware complexity and increases the required field of view of the optics. 

5 The logic required for the single node 30 (FIG. 7) is shown in detail in FIGS. 27-36. Assuming single flip- 

flops are used for each chute In each node and assuming that simple 2-lnput logic gates (ANDs, ORs, NANDs, 
and NORs) with fanout values less than eight are used within the design, then the resulting design requires 
a maximum of [(32m/n)+25log2(N/n)+207] logic gates per node, (2m/n)+2 detectors and {2m/n)+2 nrKxlulators. 
The detectors F1. F4. FIG. 27; F9, FIG. 30; F10, FIG. 32) and modulators (F5. F8. FIG. 29; F11, F12. FIG. 36) 

10 are implemented using FET-SEED technology. FIG. 5 shows a packet format that gives each node within a 
particular node-stage enough Information to route the packet to the correct node in the next node-stage. Thus, 
each packet has a header prepended to the raw data that can be used by the nodes to identify the output port 
to which the packet Is destined. For a growable packet switch with N output ports, this header contains 
iog2(N)+1 bits, where the first bit of the header field is the activity bit and the remaining log2(N) bits specify 

IS the destination address for the packet- I.e., the packet's desired output port. The activity bit is a logic one for 
an active packet and a logic zero for an inactive packet. As discussed later herein, network nodes are of two 
types: global and local. The path hunt logic 43 circuitry shown in FIGS. 35 and 36 is for a local node and im- 
plements the routing method of FIGS. 12-15; a global node (not shown) Implements the routing method of 
FIGS. 8-1 1 . The signal INIT Is used to preset all of the SR latches in FIG. 28 to known states so that their outputs 

20 indicate that all of the chutes are Idle. This INIT signal is asserted on all of the nodes in all of the stages at 
the beginning of each packet interval prior to the introduction of packets into the first stage of the network. 
The LOAD signal Is used to store the values of ACR and BCR (indicating when packet Aand B are in the correct 
row). R(A) and R(B) (indicating the routing bits for the packets), and AACT and BACT (indicating the activity 
bits for the packets). These values are stored in flip-flops when the header field from the packet has been 

25 shifted into the node. The CLK signal is used to synchronize the data flow within all of the nodes, because it 
Is used to shift bits into the node shift registers. 

Packet Routing Alflorlthm 

30 To route a packet through the node-stages of a Banyan network, a particular node must first determine 

if the incoming packet is active- i.e., if the activity bit is set to a logic one. If so, then the node only needs to 
compare one bit of the binary destination address to one bit of the node's own binary address to calculate the 
appropriate output link to which the packet should be routed. The bits of interest within the destination address 
and the node address are referred to as the destination pivot bit and the node pivot bit, respectively. For a 

35 network with L=N inputs, the binary destination address is represented by (Pn.Pk2. -.Pi.Po). where l=log2(N). 
The binary node address is similarly represented by (h|.i,h|.2.-.hi). The destination pivot bit for a packet in 
node-stage I is then defined to be pv, where V is the connection variable (Eq. 3) defined for the link-stage 1. 
which follows node-stage 1. The node pivot bit for a node in node-stage i is similarly defined to be hy- A routing 
bit (R) can be calculated for every packet within node-stage 1 based on these pivot bits, and the routing bit Is 

40 given by R=pv XOR hy. If the routing bit R=0. then the packet is routed to the straight output linkfrom the node, 
which provides a connection to the next node-stage defined by the mapping function B|0 (Eq. 2). If the routing 
bit R=1, then the packet is routed to the angled output linkfrom the node, which provides a connection to the 
next node- stage defined by the mapping functfon B,^ (Eq. 2). As an example, the single packet in FIG. 5 orig- 
inating at input 8 and destined for output 5 has a destination address given by pa ,P2 ,Pi ,Po=0101. The destin- 

45 atlon pivot bit in node 4 of node-stage 0 is Pa^O and the node pivot bit is hs^l, so the routing bit R^l and the 
packet is routed on the angled output link. The destination pivot bit in node 0 of node-stage 1 1s P2=1 and the 
node pivot bit is h2=0, so the routing bit R=1 and the packet is again routed on the angled output link. Repeating 
this algorithm in the remaining node-stages results In the packet being routed to its desired output port (Note: 
For the last stage in FIG. 5, the value of ho is assumed to be 0, and the "straight" output link Is the top output 

50 while the "angled" output link is the bottom output). In FIGS. 27-36, the bits of a packet received on input A 
are designated by A1 A(log2[N/n])- and the bits of a packet received on Input B are 'designated by B1 - 
B(log2[N/nD • The bits of the node number are designated by hi — h(log2[N/n]). 

Transferring a Packet to the Chute-Multlplexing Sub-Network 

55 

Within Canyan distribution network 23, a node determines when a packet should be transferred from the 
switching sub-network to the chute-multiplexing sut>-network by examining the most-significant log2(N/n) bits 
of the destlnatton address. If these bits match the binary address of the output packet module (00, 01 , 10 or 
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11) to which the node Is connected, then they will also match the most-significant log2(N/n) bits of the node 
address, and the node will transfer the packet to an available chute within the chute-multiplexing sub-network 
(if an available chute exists). This is shown in the Canyan distribution network 23 of FIG. 6. where N=16 and 
n=4. The two most-significant bits of the packet destination address (P3,P2.Pi.Po=0101) match the two most- 
5 significant bits of node Oil (node 3) when the packet passes through node-stage 2. and they also match the 
binary address of the desired output packet module (module 01) to which that node is connected. 

Contention Resolution; Packet Blocking, and Packet Deflection 

10 In a growable packet switch arrangement with a Canyan distribution network, there are several reasons 

why a packet may be kept firom propagating along its desired path. These reasons can be classified into two 
categories: packet deflection and packet blocking. Packet deflection and packet blocking are different, because 
a deflected packet is merely re-routed along a sub-optimal path, but it still has a chance of being routed to its 
desired output port. However, a packet that has been blocked has no way to reach its desired output within 

15 the current packet slot. 

Packet deflection will occur within the switching sub-network for two reasons. First, if two packets arrive 
at a node and attempt to use the same link emanating from that node, then only one packet can be routed on 
the desired link, and the other packet is deflected and re-routed on the second unused link. Packet deflection 
can occur In a second way if a packet arrives at a node connected to its desired output packet module and 

20 finds that all of the 2m/n output chutes for that node are already occupied by other packets. Since the packet 
cannot be transferred to the chute-multiplexing network within that node, it must be deflected and rerouted 
out on the Banyan links to try to find vacant output chutes within another node in a later node-stage. 

Packet blocking can occur within the switching sub-network in several different ways. First, if alt of the m 
chutes connected to the desired output packet module are already occupied by other packets, then a packet 

25 attempting to reach that output packet module is blocked. This can happen if the desired output packet module 
is a "hot-spot" whose outputs have many packets destined for them. Packet blocking can also occur If a packet 
is deflected so many times that it is not able to locate a node with an Idle output chute before reaching the last 
node-stage in the switching sutnnetwork. This second type of packet blocking can occur even if some of the 
output chutes attached to the desired output packet module are idle. A third type of packet blocking can occur 

30 if a packet is routed to an idle output chute, but the FIFO queue attached to that output chute is full. This last 
type of packet blocking is not analyzed herein because queueing theory models can be used to determine 
queue lengths that minimize buffer overflows and yield sufficiently low packet loss probabilities. Infinite-length 
FIFO queues are assumed herein, so the third type of packet blocking does not occur. 

The hardware within a node cannot solve the problem of packet blocking, but It can Implement a contention 

35 resolution algorithm to determine which packet receives priority when two packets are destined for the same 
output link or when a packet tries to enter a set of occupied output chutes. When packet contention arises and 
the contention resolution algorithm is used, one of the packets is deflected onto a sub-optimal path within the 
switching sub-network. 

in a Canyan network, there are two basic types of link-stages created by the partitions of the output packet 

40 modules (global link-stages and local link-stages), and different contention resolution algorithms are applied 
to these different types of link-stages. In general, a global link-stage provides connections between nodes that 
are connected (via output chutes) to different output packet modules, while a local link-stage provides connec- 
tions between nodes that are connected (via output chutes) to the same output packet module. If the connection 
variable (V) fora particular I ink-stage is greater than log2(n)-1 . then the link-stage provides global connections. 

45 If the connection variable (V) for a particular link-stage is less than or equal to log2(n)-1 , then the link-stage 
provides local connections. Thus, In the network of FIG. 6, all of the link-stages with V^l provide global con- 
nections, while the link-stages with V^1 provide local connections. 

For convenience, a global node-stage is defined as a node-stage that precedes a global link-stage, and 
a local node-stage is defined as a node-stage that precedes a local link-stage. FIGS. 8-11 comprise a flow- 

50 chart forthe contention resolution algorithm (routing method) used by a node in a global node-stage, and FIGS. 
12-15 comprise a flow-chart for the contention resolution algorithm (routing method) used by a node in a local 
node-stage. FIGS. 8-11 indicate that for global node-stages, input A (arriving from the straight link-stage con- 
nection) takes precedence over input B (arriving from the angled link-stage connection). This guarantees that 
a packet that has arrived at its desired output row but has been deflected by occupied output chutes will con- 

55 tinue to propagate on the straight paths and stay in that output row until it arrives at the local link-stages. In 
the local link-stages, the deflected packet is transferred to another output row associated with its desired out- 
put packet module, where an Idle output chute in that row is located (if there is one). FIGS. 12-15 indicate that 
for local node-stages, input A also takes precedence over input B. If either packet is deflected due to occupied 
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output chutes, it is always routed on the angied link-stage connection to permit it to search for an idle output 
chute in another output row. 

Since different contention resolution algorithms are used in local and global node-stages, the path hunt 
logic for nodes in local node-stages is different than the path hunt logic for nodes in global node-stages. Both 
5 sets of logic can be combined if another signal (identifying the type of node-stage) is supplied to each node, 
but this results in more complicated node logic. It is assumed herein that two different device array designs 
are used to provide different path hunt logic for the two types of node-stages. 

Performance Analysis of the Canyan Distribution Network 

10 

Because the structure of the Canyan distribution network is architecturally different from most of the other 
networks that have been proposed for use within a packet-switched environment, its operational characteris- 
tics are determined. Three different performance metrics are calculated: the network delay, the network 
throughput, and the packet loss probability. 

IS 

Network Delay 

The delay that a packet encounters as it passes through the Canyan distribution network is dependent 
on the node-stage in which the packet is transferred from the switching sub-network to the chute-multiplexing 

20 sub-network. While in the switching sut>-network, t he packet encounters log2(N/n)-i-4 bit periods of delay within 
each node-stage. Once it is transferred to the chute-multiplexing sub-network, the packet encounters one bit 
period of delay within each node-stage. Thus, if a particular Canyan network design requires S node-stages, 
then the minimum delay within the network is given by iog2(N/n)'t'4i-(S-1) bit periods, while the maximum delay 
is given by (S-1)Pog2(N/n)+4]+1 bit periods. These differences in delays require (S-2)[log2(N/n)'i-3]-bit guard- 

25 band intervals to be inserted between adjacent packet slots, which requires a slight speed-up of the data rates 
within the distribution network. Differences between different packets propagating within the network in a giv- 
en packet slot can then be equalized at the FIFO queues. 

Network Throughput and Packet Loss Probability 

30 

The number of node-stages (S) required for a particular application is typically determined by the networic 
throughput and the required packet loss probability. Two different analyses were used to study these network 
characteristics within the Canyan distribution network: a computer simulation and an analytical model based 
on a Markov chain. 

35 The analytical model was based on an approach in which the packet load in node-stage i is determined 

using the packet load in node-stage i-1. Thus, if the packet load entering the switch at node-stage 0 is known, 
then the packet load at any node-stage within the switch can be subsequently calculated. 

Several assumptions have been made to simplify the Markov analysis. First, the Canyan distribution net- 
work Is assumed to support L inputs and (Nm/n) outputs, and each of the (N/n) output packet modules is as- 

40 sumed to support m Inputs and n outputs. Thus, the chute-multlplexing portion of each switching node has 
C=(2m/n) output chutes. The packets arriving at the network Inputs are assumed to be synchronous, and the 
arriving packets are assumed to be random and independent. The destinations for the packets are also as- 
sumed to be uniformly distributed over all N of the output ports emanating from the output packet modules. 
For any packet within node-stage I of the Canyan distribution network, a feature known as the hop distance 

45 D is defined to be the number of global link-stages through which a packet must pass before It arrives In a 
node-stage where It could be transferred to one of its desired output chutes. The hop distance D for a packet 
passing through the Canyan distribution network will remain unchanged from the input of node-stage i to the 
output of link-stage I. Thus, the hop distance D Is assumed to change at the interface between link-stage i and 
node-stage i-i-l . For a packet entering node-stage 0 of a Canyan network, the hop distance D is given by some 

so value between 0 and X=log2(N/n). Whenever a packet is routed along one of its desired global link-stage con- 
nections, its hop distance is decreased by one as it passes through the interface between the link-stage and 
the next node-stage. If deflected in a global link-stage, the hop distance is always reset to D=X'i'1=log2(N/n)'i'1. 
Whenever a packet's hop distance reaches zero, It can be transferred to an idle output chute (if one exists). If 
an idle output chute does not exist, then the packet's hop distance is set to D=-1 (indicating that its hop distance 

55 is actually zero, but it has been deflected by occupied output chutes and has no chance of being transferred 
to an output chute until It changes rows in a local link-stage). Thus, it is not possible to have a hop distance of 
zero in a global link-stage. When a packet with a hop distance of D=-1 moves into a local node-stage, its hop 
distance can be changed back to zero if it is sent out on the angled link. For all hop distances greater than 
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zero, the hop distance for a particular packet is never changed when the packet has passed through a local 
link-stage. Thus, for packets In any link-stage of the Canyan network, the possible hop distance values will 
always fall in the range from -1 toX+1=log2(N/n)+1. The hop distances for several packets propagating through 
the stages of a Canyan distribution network are illustrated in FIG. 16. The hop distance of a packet is merely 
a logical feature that is useful in the Markov performance analysis. The actual node hardware of FIG. 7 does 
not explicitly use the hop distance to determine the required path for the propagating packets, and the hop 
distance is never explteltly carried within the header field of the packets. Thus, the hop distances shown in 
the packet headers of FIG. 16 are merely logical values that can always be determined from a knowledge of 
the packet's destination address and the node number and the connection variable for the node in which the 
packet resides. 

For the analysis of the throughput and packet loss probabilities, the offered load to the network (the average 
fraction of input ports with active packets during a given packet slot) is defined as pt. where 0<pt^1 . Y(D,i) repre- 
sents the state that a packet is in when it is in a link in link-stage i wit h a hop distance of D (-1 ^[^X+1 =log2(N/n)+1 ); 
Z(R,i) represents the state that a packet Is in when It Is in output chute R in link-stage i (0<R^C-1=(2m/n)-1). 
Thus, there are X+C+2=log2(N/n)+(2m/n)+2 different states in which a packet can exist in a particular link- 
stage. P[Y(D,i)] represents the probability that packets are in a link in link-stage i with a hop distance of D, 
and P[Z(R,i}] represents the probability that packets are in a chute in link-stage i and that the chute has the 
chute number R. From the definitions above, it follows that the following relation Is satisfied for every stage I 
within the Canyan network: 

Pl== P[Y(D.i)l+^£ P[Z(R,i)] u} 

In addition. In any stage I, the percentage of all L possible input packets that have been routed to one of their 
desired output chutes (the network throughput or carried load) is given by: 

C-l 

throughputs S P[Z(R,i)] [51 

and the percentage of all L possible input packets that have not yet been routed to one of their desired output 
chutes is given by 1 -(throughput). Thus, if a Canyan distribution network is designed with exactly S node- 
stages, then the packet loss probability for the distribution network is given by: 

P[packct loss]=(l-^£ P[Z(R.S)J)/pL [61 

R-O 



Initial conditions for the Markov analysis can be determined by examining the packetsbeforethey actually enter 
the Canyan network. It can be shown that for a packet entering the first node-stage of a Canyan network (whteh 
is therefore In link-stage -1), the probability that the packet has a hop distance of D (-1^D£X^1=log2(N/n)+1) is 
given by: 

P[Y(- 1,-1)1 = 0, forD= -1 [7] 
P[Y(0, -1)] = n/N, forD = 0 
P[Y(D, - 1)] = 2° - in/N, for I^D^ = log2(N/n) 
PIY(X + 1, - 1)1 = 0, for D = X + 1 = log2(N/n) + 1 
In addition, the chutes are Initially empty in link-stage -1. so it follows that 

P[Z(R. - 1 )] = 0, for O^R^C - 1 = (2m/n) - 1 [8] 
For the Markov chain model, a packet in a particular state in link-stage i-1 will always transition to one of 
theX+C+2=log2(N/n)+(2m/n)+2 states in link-stage i. We must therefore define the transition probabilities (or 
conditional probabilities) that take a packet from one state in link-stage i-1 to another state in link-stage i. For 
example, the transition probability P Y(3,i)/Y(4,i-1)] defines the transition probability that a packet in iink-stage 
1-1 with a hop distance of will eventually have a hop distance of D=3 when It passes through link-stage I. 
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stated another way, P[Y(3,i)/Y(4J-1)] gives the probability that a packet will have a hop distance of D=3 in 
link>stage i given that the packet had a hop distance of D=4 in link-stage i-1. Based on the definition of the 
transition probabilities, the following relation must be satisfied for any state Y(E,i-1 ) in link-stage i-1: 



X+1 C-1 

£ P[Y(D.i)/Y(E.i-l)+ Z P[Z(R.i)/Y(E,i-l)=1.0 [9] 

where E is the hop distance ranging fronn -1 to X+1=log2(N/n)'i-1. The following relation must be satisfied for 
any state Z(FJ-1) In link-stage 1-1: 

P[Y(D.i)/Z(F.i-l)+^£P[Z(R.i)/Z(F.i-l)=1.0 [101 



20 where F is the chute number ranging from 0 to C-1=(2m/n)-1. Eq. 7 and Eq. 8 merely state that a packet in a 
particular state in link-stage 1-1 must transition (with probability 1.0) to one of the states in link-stage i. 

Once all of the transition probabilities have been determined, the probability of being in a particular state 
in link-stage I is determined from the probabilities of being In each of the possible states in link-stage i-1 and 
from the associated transition probabilities. In particular, the probability of being in state Y(E,i) in link-stage i 

25 is calculated using the theoiBm of total probability: 



P[Y(E,i)l= ^£ PlY(D.i-l)] PIY(E.i)/Y(D.i-l)] + 

30 ^ ^ D=-l 

± P[(Z(R.i-l)] P[Y(E,i)/Z(R.i-l)] rii] 

35 The probability of being In state Z(F,I) In link-stage i Is also calculated using the theorem of total probability: 

P[Z(F,i)l= ^£ P[Y(D,i-l)] PtZ(F.i)/Y(D.i-l)l + 

£ P[(Z(Ra-l)l P[Z(F.i)/Z(R.i-l)l [12] 
R»0 

45 The transition probabilities describe an event that occurs at the interface between link-stage i-1 and node- 

stage I (as shown in FIG. 16). The values for these transition probabilities are very dependent on the type of 
link-stages (global or local) that precede and follow node-stage i. In addition, another set of transition prob- 
abilities is defined for the first node-stage which is preceded by the network inputs. Thus, there are five dif- 
ferent types of node-stages for which transition probabilities are defined: node-stages preceded by network 

so inputs and followed by a global link-stage (input-global node-stages); node-stages preceded by a global link- 
stage and followed by a global link-stage (global-global node-stages); node-stages preceded by a global link- 
stage and fbllowed by a local link-stage (global-local node-stages); node-stages preceded by a local link-stage 
and followed by a local link-stage (local-local node-stages); and node-stages preceded by a local link-stage 
and followed by a global link-stage (local-global node-stages). 

55 To simplify the Markov analysis, another useful packet state known as Ztot(i) is defined for link-stage i as 

the union of ail of the different states In which a packet is in an output chute. Thus, it follows that the probabflity 
of being in state Ztat(i) is given by: 
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P[Z», (i)] =^2 P[(Z(R.i) J 113] 

Once P[Ziot(i)] has been determined (using the transition probabilities below), the values for P[Z(R,i)] can then 
be determined for each value of R (O^R^C-1). To do this, it is assumed that the chutes are filled In ascending 
order, so the first packet sent into the chutes in a particular row goes into chute R=0, and the second packet 
goes into chute R=1 , etc. Given this assumption. It can be shown that 

P[Z(0.i)]-P[2(0.i-l)l+P(Z„^(i)]jl-P[Z(0.i-l)lj. for R=0, 
P[Z(R.i)]=P[Z(R.i-l)l + 

P[Ztot(i)] n^P[(Z(k,i-l)l|l-P[Z^^^^ forO^R^C-2. 

p[Z(c- i.i)]=P[Z(C- la- 1)1+ 

P[Ztot(i)l ri P[(Z(ka- l))» for R=C-1. [14] 

These formulae indicate that the increase in the probability of chute R being occupied by a packet is directly 
proportional to 1) the probability that packets are being routed to the chutes (P[Z|at(i)]), 2) the prot>ability that 
all of the chutes with values less than R are occupied 

(*np[z(k.i-i)D 

k-O 

and 3) the probability that chute R is idle (1>P[Z(R,i<1 }]). The third term is not needed for the last chute, because 
the transition probabilities described below do not increase the value of P[Z|ot(l)] if the last chute Is already 
occupied. 

Given the background information described above, fbrmulae are developed for all of the transltton prob- 
abilities within the Canyan distribution network. The non-zero transition probabilities for each of the five afore- 
mentioned node-stage types are described In detaD below. 

Input-Global Node-Stefles, 

There are four non-zero transition probabilities that exist for input-global node-stages. 

P[Y(D,0)/Y(P,>»1)1, for I^D^ In Input-Globar Node-Stages. 

Assume packet #1 arrives at a node in node-stage 0 with 1^D^. Its hop distance remains unaltered if 
any one of the following conditions is true: 

1. Packet #1 is the only packet entering the node: 

probabiKty«l- £ P[Y(D.-1)]. 
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2. Packet #1 enters the node with packet #2. packet #2 has a hop distance D^1 , and no contention occurs 
for an output link: 

5 probabmty=:(0.5)*£* P[Y(k,- 1)]. 

kal 

3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^1 , contention occurs for an 
10 output link, and packet #1 wins: 

prDbabmty=(a5X0.5)^£ P[ Y(k, - 1 )1. 

IS 

4. Packet#1 enters the node with packet #2. packet#2 has a hop distance D=0 (so packet #2 is transferred 
to an output chute and packet #1 is routed to its desired output link): 

probability = P[Y(0,- 1)]. 

20 Since these four conditions are mutually exclusive, the total transition probability Is merely given by the sum 
of the four probabilities described above. 

P[Y(X*1,0)/Y(P,-1)1, for I^D^ In Input-Global Node-Stages. 

25 Assume packet #1 arrives at a node in node-stage 0 with 1^D^ It is deflected and its hop distance is 

reset to X*^ if the following condition is true: 

1 . Packet #1 enters the node with packet #2, packet #2 has a hop distance D^1 , contention occurs for 
an output link, and packet #1 loses: 

^ probabmiy«(0.5X0.5)^£^ P[ Y(k, - 1 )]. 

k3] 



35 P[2:tet(0)/Y(0.-1H, for D°0 In Input-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage 0 with D=0. It is transferred to an output chute if the 
following condition is true: 

1 . Packet #1 is the only packet entering the node: 

40 

probabilityftl- £ P[Y(k.-l)]. 
k»0 

45 2. Packet #1 enters the node with packet #2. packet #2 has a hop distance D^1 , so packet #2 is routed 

to Its desired output link and packet #1 Is transferred to an output chute: 



^ probabiUty=^£ P[ Y(k.- 1 )1. 

kail 

3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=0, contention occurs for an 
output chute, and packet #1 wins: 
55 probability = (0.5)P[ Y(0, - 1 )1. 

The total transition probability Is given by the sum of these three probabilities. 
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P[Y(-1,0)/Y(0r1)], for P=0 In Input-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage 0 with D=0. It is deflected and Its hop distance is reset 
to X-t-1 if the following condition is true: 

1. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=0, contention occurs for 
an output chute, and packet #1 loses: 

probability = (0.5)P[Y(0, - 1)] 

Global-Global Node-Stages, 

There are eight non-zero transition probabilities that exist for global-global node-stages. 

P[Y(D-1J)/Y(D,M)l,fof 3^D^-H In Global-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with 3^D^+1. Its hop distance is then decremented 
by one if any one of the following conditions is true: 

1 . Packet #1 is the only packet entering the node: 

piobahffity=l.P[Y(-l,M)h^£ P[Y(k.i- 1)1. 

2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^, and no contention occurs 
for an output link: 

probabiIity=(a5)^£^ P( Y(k,i- 1 )]. 
ks2 



3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^2, contention occurs for an 
output link, and packet #1 wins: 



pn>bahaiiys-(0,5)(0.5)^£ P[ Y(k.i- 1 )1. 

ka2 



4. Packet #1 enters the node with packet if2, packet #2 has a hop distance D=1, packet #2 is deflected 
to the straight link because all of the output chutes (including chute C-1) are already occupied, and packet 
#1 wanted to use the cross link: 

probability = (0.5)P[Y(1.i - 1)]P[Z(C - 1,1 - 1)]. 

5. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=1, and packet #2 is trans- 
ferred to the output chutes because there is an idle one available, so packet #1 can be routed to its desired 
output link: 

probability = P[Y(1,i - 1)][1 - P[Z(C - 1,i - 1)]]. 

6. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 
deflected and will continue out on the straight link), and packet #1 wanted to use the cross link: 

probability = (0.5)P[Y( - 1.1 - 1)] 
The total transition probability is given by the sum of the six probabilities described above. 

P[Y(X-H,l)/Y(D,i-1)], for 3^D^-H In Global-Global Node-Stages, 

Assume packet #1 arrives at a node in node-stage i with 3^D^+1. It is deflected and its hop distance is 
reset to X-i-l if the any one of the following conditions is true: 

1 . Packet #1 enters the node with packet #2, packet #2 has a hop distance D^2. contention occurs for an 
output link, and packet #1 loses: 
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probability=(0.5)(a5)^£ P[Y(k,i- 1 )]. 

5 

2. Packet #1 enters the node with packet #2. packet #2 has a hop distance D=1, packet #2 is defiected 
to the straight link because all of the output chutes (including chute C-1) are already occupied, and packet 
#1 wanted to use the straight link: 

probability = (0.5)P[Y(1.i - 1)]P[Z(C - 1.i - 1)]. 
10 3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 

deflected and will continue out on the straight link), and packet #1 wanted to use the straight link: 

probability = {0.5)P[Y( - 1,i - 1)] 
The total transition probability is given by the sum of the three probabilities described above. 

15 P[Y(1,I)/Y(2,I-1)], for D g 2 In Global-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=2. Its hop distance is then decremented to 
. D=1 and it is routed out on its desired cross link If any one of the following oonditbns Is true: 

1. Packet #1 Is the only packet entering the node: 

20 

prDbahility»l.P[Y(-l.i-l)].^£* P[Y(k.i- 1)1. 

2S 

2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^3, and no contentk>n occurs 
for an output link (indicating packet #2 wanted the straight link): 

X+l 

pn>bability=(0.5) J P[Y(k.i- 1 )]. 

k-3 



3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^3, contention occurs for 
the cross link, and packet #1 wins: 

35 

pn>bability^(0.5)(a5)^2 P[ Y(k.i - 1 )]. 

k«3 

40 4. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=2 (indicating packet #2 Is 

also destined for the cross link), contention always occurs, and packet #1 wins: 

probability = (0.5)P[Y(2,i - 1)] 

5. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=1, packet #2 is deflected 
to the straight link because all of the output chutes (including chute 01) are already occupied, so packet 

45 #1 can use Its desired cross link: 

probability = PIY(1 ,i - 1)]P[Z(C - 1,1 - 1)]. 

6. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=1, and packet #2 is trans- 
ferred to the output chutes because there Is an idle one available, so packet #1 can be routed to its desired 
cross link: 

50 probability = PIY(1,i - 1)][1 - P[Z(C - 1,i - 1)]]. 

7. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1 , (Indicating It has been 
deflected and will continue out on the straight link), so packet #1 can be routed to Its desired cross link: 

probability = P[Y(- 1.1-1)] 
The total transition probability is given by the sum of these seven probabilities. 

55 

P[Y(X-i-1,l)/Y(2,l-1)], for D»2 In Global-Global Node-Stages. 



Assume packet #1 arrives at a node in node-stage i with D=2. It is deflected and its hop distance is reset 
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to if any one of the following conditions is true: 

1 . Packet #1 enters the node with packet #2, packet #2 has a hop distance D^3, contention occurs for 
the cross link, and packet #1 loses: 



probabUity=(0.5)(0.5)'£* P[ Y(k.i - 1 )]. 

10 2. Packet #1 enters the node with packet #2. packet #2 has a hop distance D=2 (indicating packet #2 is 

also destined for the cross link), contention always occurs, and packet #1 loses: 

probability = (0.5)P[Y(2,i - 1)] 
The total transition probability is given by the sum of these two probabilities. 

IS P[Y(-1,i)/[Y(1,l-1)], for Dal In Global-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=1. (Note: It is not possible for two packets to 
arrive at a node with D=1 for both packets, because both packets would have to arrive on the cross path). It 
is deflected to the straight path and its hop distance is set to -1 if any one of the following conditions Is true: 
20 1 . Packet #1 Is the only packet entering the node, and packet #1 Is deflected to the straight link because 

all of the output chutes (including chute C-1) are already occupied: 

probability=|l--P[Y(-la-l)l/P[Z(C-ia-l)l-P[Y(0,i-l)]«^^^ 



25 



50 



P(Z(C-U-1)]. 



2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^O, and packet #1 is deflected 
30 to the straight link because all of the output chutes (including chute C-1) are already occupied: 

probabiHty«jP[Y(0,i-l)]+^£VlY(k,i-l)llpiZ(C-U-l)L 

35 I k-2 J 



The total transition probability is given by the sum of these two probabilities. 

40 P[Y(X*1,l)/Y(1,l-1)l,f6rD°1 In Global-Global Node-Stages, 

Assume packet #1 arrives at a node in node-stage i with D=1 . It Is deflected and its hop distance is reset 
to X+l if the following condition is true: 

1. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 
45 deflected and will continue out on the straight link), so packet #1 is deflected to the cross link because all of 
the output chutes (Including chute C-1) are obviously already occupied: 

probability = P[Y(- 1.1- 1)] 



P[2tot(0)/Y(i,l-1)l,for P"1 In Global-Global Node-Stages. 



Assume packet #1 arrives at a node in node-stage I with D=1. It Is transferred to an output chute If any 
one of the following conditions is true: 

1 . Packet #1 is the only packet entering the node and packet #1 is transferred to the output chutes because 
there is an idle one available (Note: It is not possible for there to be an idle output chute in a node and for 
55 a packet to arrive at that node with D=-1): 
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piobaWUty^jl -P[ Y(0.i- D] - PIY(k,i- 1 )l| |i .p[Z(C- 1 .i- 1 )] j. 

2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^O, so packet #2 is routed 
to its desired output link and packet #1 Is transferred to an klle output chute: 

pn>babmty=jp[Y(0.i-l)l + J£PlY(k.i-l)jj jl-PtZ(C-l.i-l)j|- 



The total transition probability is given by the sum of these two probabilities. 
P[Y(-1,I)/YM,H)1, for D=-1 In Global-Global Node-Sfaifles. 



Assume packet #1 arrives at a node in node-stage i with D=-1 (indicating it arrives on the straight input 
link). It is passed through to the straight output link and its hop distance remains at D=-1 because the straight 
20 input always has priority: 

probability s 1. 

GlobaI'Local Node«Stage8. 

25 There are five non-zero transition probabilities that exist for global-local node-stages. 

P[Y(P-1J)/Y(D>1)1, for 2^DgX-H in Global-Local Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with 2^D^X+1. Its hop distance is then decremented 
30 by one because It doesn't matter which output link the packets are sent to in the local link-stage that follows: 

probability = 1. 

P[Y(-1,I)/Y(1,M)1, forPol In Global-Local Node-Stafles. 

35 Assume packet #1 arrives at a node in node-stage i with D=1. It is deflected to the straight path and Its 

hop distance is set to -1 if the following condition is true: 

1. Pacl(et#1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 
deflected and will continue out on the cross link within the local link-stage that follows), so packet #1 is de- 
flected to the straight link because all of the output chutes (Including chute G-1) are obviously already occupied: 

40 probability = PIY( - 1 ,1 - 1 )] 

P[Y(0,I)/Y(1, 1-1)1, for 0=1 in Global-Local Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=1 . It is deflected and its hop distance is reset 
46 to 0 (indicating that it Is deflected by full chutes in node-stage i, but it has a chance of being transferred to an 
Idle chute in node-stage i+1) if any one of the following conditions Is true: 

1 . Packet #1 is the only packet entering the node, and packet #1 Is deflected to the cross link because all 
of the output chutes (including chute C-1) are already occupied: 



pn>babmty=jl-P[Y(-ia^l)]/P[2(C-ia-l)l-PlY(0J-l)J^ 



PIZ(C.U-1)1- 



2. Packet #1 enters the node with packet#2, packet #2 has a hop distance D^O (it is not possible for packet 
#2 to have a hop distance of D=0 or D=1 in this node-stage) and packet #1 is deflected to the cross link 
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because all of the output chutes (including chute C-1) are already occupied: 
probabmty=|p[Y(0.i-l)l + J£ P[Y(^ 

The total transition probability Is given by the sum of these two probabilities. 
P[ZtetY(0VV(1r 1)1. for D°1 in Global- Local Node- Stages. 



Assume packet #1 arrives at a node in node-stage i with D=1. It is transferred to an output chute If any 
one of the following conditions is true: 
IS 1 . Packet #1 is the only packet entering the node and packet #1 is transferred to the output chutes because 

there Is an Idle one available (Note: It is not possible for there to be an Idle output chute in a node and for 
a packet to arrive at that node with D=-1, and it is also not possible for two packets to arrive at the same 
node with D=1): 

probabmty=|l-P[Y(Oa-l)J-JEPW^ |l-P[Z(C-l.i-l)]J. 

25 2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^O. so packet #2 is routed 

to its desired output link and packet #1 is transferred to an kJle output chute: 

^ probahmiy»|p[Y(04-l)l+^2 P[Y(k.i-l)]| |l-P(Z(C-l.i-l)]|. 



The total transition probability is given by the sum of these two probabilities. 

P[Y(0,i)/Y(-1,l-1)],forD"-1 in Global-Local Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D^1 (indicating it arrives on the straight input 
link). It is passed through to the cross output link and its hop distance is set to 0 if the following conditton is 
40 true: 

1. Always, because the straight input always has priority: 

probability =1. 



Local-Local Node-Stages. 

There are f h/e non-zero transition probabilities that exist for local-local node-stages. 

P[Y(D,l)/Y(P,l-1)],for1gD^-H In Local-Local Node-Stages. 

so Assume packet #1 arrives at a node in node-stage i with I^D^-i-l. Its hop distance remains unaltered 

because it doesn't matter which output link the packets are sent to in the local link-stage that follows: prob- 
abiiity=1. 



PIY(-1,D/Y(0,M)l,forD=0 in Local-Local Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=0 (indicating that it arrives on the cross input 
link), it is deflected to the straight path and its hop distance is set to -1 if the following conditton is b-ue: 

1 . Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1 , (indicating it has been 
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deflected and will continue out on the cross link within the local link-stage that follows), so packet #1 is de- 
flected to the straight link because all of the output chutes (including chute C-1 ) are obviously already occupied: 

probability = P[Y(-1,i-1)] 

5 P[Y(0,i)/Y(0,l-1)l, for DaQ in Local-Local Node-Stafles. 

Assume packet #1 arrives at a node in node-stage I with D=0. It is deflected and its hop distance is reset 
to 0 (indicating that It is deflected by full chutes in node-stage i, but It has a chance of being transferred to an 
idle chute in node-stage i-i-l) if any one of the foilowing conditions is true: 
10 1 . Packet #1 is the only packet entering the node, and packet #1 is deflected to the cross link because all 

of the output chutes (including chute C-1) are already occupied. (Note: It is not possible for two packets 

to arrive at a node with D=0): 

15 in^babmty=]l-P(Y(--ia-l)l/PIZ(C-l,i^l)l-^£PIY(k.i-l)] 
P[Z(C-l4-l)]- 




2. Packet#1 enters the node with packet #2. packet #2 has a hop distance D^O (it is not possible for packet 
#2 to also have a hop distance of D=0 in this node-stage), and packet #1 is deflected to the cross link 
because all of the output chutes (including chute C-1) are already occupied: 

probabiIity= 2 P[Y(k.i-l)lP[Z(C.I4-l)I. 



30 P[Z|,tY(0yY(0r 1)], for D«0 In Local- Local Node- Stages. 

Assunne packet #1 arrives at a node in node-stage i with D=0. It is transferred to an output chute if any 
one of the following conditions is true: 

1 . Packet #1 is the only packet entering the node and packet#1 is transferred to the output chutes because 
35 there is an idle one available (Note: It is not possible for there to be an idle output chute in a node and for 

a packet to arrive at that node with D=-1, and it is also not possible for two packets to arrive at the same 
node with D=0): 



40 



so 



55 



piobabUity=|l -^2 P{Y(k.i- DjJ |i -P[Z(C- l.i- 



2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^O, so packet #2 is routed 
45 to its desired output link and packet #1 is transferred to an idle output chute: 

piobability=| £V[Y(k,i- 1 )]| ji -PtZ(C- 1 .i- 1 )]|. 



The total transition probability is given by the sum of these two probabilities. 

P[Y(0JVY(-1,l-1)l,f6rDo-1 In Local-Local Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=-1 (indicating it arrives on the straight Input 
link). It is passed through to the cross output link and its hop distance is set to 0 because the straight input 
always has priority: 
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probability = 1 . 

Local"Global Node-Stagas, 

There are eight non-zero transition probabilities that exist for local-global node-stages. 

P[Y(P,iyY(P,i-1)],fdr2^P^*1 In Local-Global Node-Stages. 

Assume packet #1 arrives at a node In node-stage I with 2^D£X'M. Its hop distance remains unaltered If 
any one of the following conditions is true: 

1 . Packet #1 Is the only packet entering the node: 

probabaity=l.^£ P[Y(k.i«l)l. 

2. Packet #1 enters the node with packet #2, packet #2 has a hop distance 0^1 , and no contention occurs 
for an output link: 

probabaity=(0.5)^£* PIY(k.i- 1)1. 

3. Packet #1 enters the node with packet #2. packet #2 has a hop distance D^l , contention occurs for an 
output link, and packet #1 wins: 

probabaiiy=(0.5)(0.5)^X^ P[ Y(k.i - 1 )1. 



4. Packet #1 enters the node with packet if2, packet #2 has a hop distance D=0. packet if2 Is deflected 
to the straight link because all of the output chutes (Including chute C-1) are already occupied, and packet 
#1 wanted to use the cross link: 

probability = (0.5)P[Y(0,I - 1 .1 - 1)]. 

5. Packet #1 enters the node with packet #2. packet #2 has a hop distance D=0, and packet #2 is trans- 
ferred to the output chutes because there Is an Idle one available, so packet #1 can be routed to its desired 
output link: 

probability = P[Y(0.1 - 1)][1 - P[Z(C - 1.1 - I)]]. 

6. Packet #1 enters the node with packet #2, packet #2 has a hop distance Ds-1, (Indicating It has been 
deflected and will continue out on the straight link), and packet #1 wanted to use the cross link: 

probability = (0.5)PIY( - 1.i - 1)] 
The total transition probability Is given by the sum of the six probabilities described above. 

P[Y(X-i-1,i)/Y(D,i-1)L for 2^0^*1 In Local«Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage I with 2^D^•^^. It Is deflected and Its hop distance is 
reset to X-i-l if the any one of the following conditions is true: 

1 . Packet #1 enters the node with packet #2, packet #2 has a hop distance D^1 , contention occurs for an 
output link, and packet #1 loses: 

' pxDbability=(a5X0.5)^£^ PI Y(k.i- 1 )]. 



2. Packet #1 enters the node with packet #2. packet #2 has a hop distance D^O, packet #2 is deflected 
to the straight link because all of the output chutes (including chute C-1) are already occupied, and packet 
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#1 wanted to use the straight link: 

probability = (0.5)P[Y(0 j - 1)]P[Z(C - 1,1 - 1)]. 
3. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 
deflected and will continue out on the straight link), and packet #1 wanted to use the straight link: 

probability = (0.6)P(Y( - 1.1 - 1)] 
The total transition piobability Is given by the sum of the three probabilities described above. 

P[Y(1J)/Y(1, 1-1)1, for Dal In Local-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage I with D=1. Its hop distance remains at D=1 and It Is 
routed out on its desired cross link if any one of the following conditions is true: 

1 . Packet #1 Is the only packet entering the node: 

X + l 

2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^, and no contention occurs 
for an output link (indicating packet #2 wanted the straight link): 

probability=(0.5)^£ P[ Y(k,i- 1 ) J. 

3. Packet #1 enters the node with packet #2, packet #2 has a hop distance contentton occurs for 
the cross link, and packet #1 wins: 

probability«(0.5)(0.5)^£ P[ Y(k,i- 1 )]. 
kB2 



4. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=1 (indicating packet #2 is 
also destined for the cross link), contention always occurs, and packet #1 wins: 

probability = (0.5)P[Y(1.i - 1)] 

5. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=0. packet #2 Is deflected 
to the straight link because all of the output chutes (including chute C-1) are already occupied, so packet 
#1 can use its desired cross link: 

probability = PIY(0,i - 1)]P[2(C - 1,i - 1)]. 

6. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=0, and packet #2 is trans- 
ferred to the output chutes because there Is an Idle one available, so packet #1 can be routed to its desired 
cross link: 

probability = P[Y(0.i - 1)][1 - P[Z(C - 1,1 - 1)]]. 

7. Packet #1 enters the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 
deflected and will continue out on the straight link), so packet #1 can be routed to Its desired cross link: 

probability = P[Y(- 1,1-1)] 
The total transition probability is given by the sum of these seven probabilities. 

P[Y(X*1,D/Y(1,l-1)l,fdrDg1 In Local-Global Node-Stages. 

Assume packet #1 arrives at a node in node-stage i with D=1. It is deflected and its hop distance is reset 
to X+1 if any one of the following conditions is true: 

1. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^2, contention occurs for 
the cross link, and packet #1 loses: 

X+l 

probability=(0.5)(a5) £ P[Y(k,i-l)l. 

k»2 
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2. Packet #1 enters the node with packet #2, packet #2 has a hop distance 0^1 (indicating packet #2 is 
also destined for the cross link), contention always occurs, and packet #1 loses: 

probability = (0.5)P[Y(1.i - 1)] 
The total transition probability Is given by the sum of these two probabilities. 

5 

P[Y(-1,I)/Y(0>M)], for Dap In Local-Global Node-Stages. 

Assume packet #1 arrives at a node In node-stage i with D=0. (Note: It is not possible for two packets to 
arrive at a node with D=0 for both packets, because both packets would have to arrive on the cross path). It 
10 is deflected to the straight path and its hop distance is set to -1 if any one of the following conditions is true: 
1 . Packet #1 is the only packet entering the node, and packet #1 is deflected to the straight link because 
all of the output chutes (including chute C-1) are already occupied: 



15 



20 



30 



probabilityojl -P[Y(- 1 )]/P[Z(C- 1 .i- 1 )] -^f P[ Y(k.i- 1 )] j 
P[Z(C-U-1)1. 

2. Packet#1 enters the node with packet #2, packet#2 has a hop distance D^1, and packet #1 Is deflected 
to the straight link because all of the output chutes (including chute C-1) are already occupied: 



pn>babiUty«|^£^P[ Y(k.i-- 1 )]j P[Z(C-14-1)]. 



The total transition probability is given by the sum of these two probabilities. 
P[Y(X'H,I)/Y(0,I-1)], for D=0 In Local-Global Node-Stages. 



Assume packet #1 arrives at a node In node-stage i with D=0. It is deflected and its hop distance is reset 
to X+1 if the following condition is true: 
35 1. Packet #1 entere the node with packet #2, packet #2 has a hop distance D=-1, (indicating it has been 

deflected and will continue out on the straight link), so packet #1 is deflected to the cross link because all of 
the output chutes (including chute C-1) are obviously already occupied: 

probability = P[Y(1 ,1-1)] 

40 P[Ztot(0)/Y(0,l-1)l, for D°0 In Local-Global Node-Stages, 

Assume packet #1 arrives at a node in node-stage i with D=0. It is transferred to an output chute if any 
one of the following conditions Is true: 

1 . Packet #1 is the only packet entering the node, and packet#1 is transferred to the output chutes because 
45 there is an idle one available. (Note: It is not possible for there to be an idle output chute in a node and 

for a packet to arrive at that node with D=-1, and it Is not possible for two packets to arrive with D=0): 

probabUi^r-jl- £*P[Y(k.i-l)]| |l-PfZ(C-l.i- 



2. Packet #1 enters the node with packet #2, packet #2 has a hop distance D^l , so packet #2 is routed 
55 to its desired output link and packet #1 is transferred to an idle output chute: 
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probahiKty=|^£ P(Y(k4-l)^ |l-.p[Z(C-l.i-l)] 

5 

The total transition probability is given by the sum of these two probabilities. 

P[Y(-1,l)/Y(-1,i-1)],for P°-1 In Local-Global Node-Stages. 

10 Assume packet #1 arrives at a node in node-stage i with D=-1 (indicating it arrives on the straight input 

link). It is passed through to the straight output link and Its hop distance remains at D=-1 because the straight 
input always has priority: probability=1. 

Canyan Network Characteristics vs.. Network Parameters 

15 

Computer simulations have been used to analyze the operating characteristics (packet loss probability, 
throughput, etc.) for Canyan distribution networks with various sizes (LxN), dimensionalities (S, m, and n), and 
offered loads (pj. The results for a network with L=N=256, m=16, n=8, and pl=1 .0 are shown in FIGS. 17-18, 
where the packet loss probability and the throughput are plotted as a function of the number of node-stages 

20 in the network (S). The shape of the resulting curves is characteristic of most Canyan networks. As expected, 
Canyan networks with more node-stages will typically have lower blocking probabilities and higher through- 
puts, because the packets are offered more opportunities to find their way to an available output chute. How- 
ever, both curves approach asymptotic limits. For example, regardless of the value of S, the packet loss prob- 
ability for the particular Canyan network in FIG. 17 never drops beiow a value of 6.8x10-^. This limit Is the 

25 growable packet switch limit of Eq. 1 , which predicts that the lowest possible blocking probability for a growable 
packet switch with m=16, n=8, and pl=1-0 is approximately 7.9x10^, which Is shown as a dashed, horizontal 
line in FIG. 1 7. In other words, for uniformly distributed destination addresses, there is a small probability that 
some of the output packet modules will have more than m=16 packets destined for them during some of the 
packet slots, so some of these packets must be dropped. (Note: It is shown that these asymptotic limits can 

30 be modified if the values of m and n are changed). 

Although the simulatk>n results provide useful information for small Canyan networks (L.N^258), simula- 
tions of larger Canyan networks typically require excessive amounts of processing power and computation 
time. As a result, the Markov model was used for the analysis of larger networks. To valkiate this model, the 
packet loss probability and the throughput for the network of FIGS. 17-18 (Ls=N=256, m=16, n=8, and pl=1.0) 

35 were re-calculated using the Markov model. The initial conditions defined by Eq. 7 and Eq. 8, Eq. 11 and Eq. 
12 and the transition probability equations described above were used to determine the network throughput 
and the packet loss probability (Eq. 5 and Eq. 6). The results are shown in FIGS. 19-20, and it can be seen 
that the curves in FIGS. 17-18 and FIG. 12 track one another very closely until the growable packet switch 
limit is reached. The Markov model does not accurately predict the existence of this limit, so a hybrid model 

40 combining the results of the Markov model and the results of the growable packet switch limit (Eq. 1) was used 
to analytically characterize the Canyan network. This hybrid model predicted the actual packet loss probability 
to be the maximum of the two packet loss probability values predicted by the Markov model and the growable 
packet switch limit The growable packet switch limit predicted for m=16, n=8, and pl=1 .0 is shown as a dashed, 
horizontal line in the packet loss probability curve of FIG. 19, and the resulting hybrid model in FIG. 19 tracks 

45 quite well with the simulation results of FIG. 17. The region with the largest error occurs at the intersection 
between the Markov model curve and the horizontal limit line, and even there, the magnitude of the error is 
still less than an order of magnitude. Similar results were observed for many different Canyan networks with 
various sizes, dimensionalities, and offered loads. 

Using the hybrid Markov model, the packet loss probabilities for many different Canyan distribution net- 

50 works were calculated. As an example, the plots in FIG. 21 indicate how the packet loss probability varies as 
a function of network size (N). These plots indicate that the growable packet switch limit remains fixed as N 
is increased (because the ratio of m and n remain unaltered), but the Markov model portion of the curves have 
different shapes and slopes. In particular, as Land N are increased, the Canyan network requires more node- 
stages to produce a desired packet loss probability. This Is a result of three effects: 1) more stages are required 

55 for full connectivity as L and N are increased. 2) a deflected packet must propagate through more stages to 
reach its desired output row as L and N are increased, and 3) there are more ways for packets to deflect one 
another as L and N are increased. 

Variations in the offered load also have an effect on the performance of Canyan networks. FIG. 22 illus- 
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trates that the same network with different loads will have different growable packet switch limits (higher loads 
have higher limits). In addition, networks with higher loads also require more node-stages to produce a desired 
packet loss probability, because there are more ways for packets to deflect one another. 

In FIG. 23. the value of m (the number of FIFO queues associated with each of the output packet modules) 
Is varied, and this only alters the growable packet switch limit. As shown in the figure, higher values of m pro- 
duce lower limits and less blocking, because there are more available output chutes providing more access to 
the output packet modules. 

FIG. 24 shows that the packet loss probability curves are significantly Influenced by changes in the value 
of n (the number of output ports associated with each output packet module). As n is increased, the growable 
packet switch limit also increases, because on the average, more packets will be homing in on and more con- 
tention will exist for the m FIFO queues associated with a particular output packet module. On the other hand, 
as n is increased, the Canyan network requires less node-stages to produce a desired packet loss probability. 
This is because the average hop distance Is lowered as n Is increased, and the number of stages through which 
a deflected packet must propagate to reach its desired output row is also decreased as n is Increased. 

The Compressed-Canyan Distribution Network 

All of the packet loss probability curves for the Canyan distribution networks in the previous section had 
simitar shapes. An interesting feature within all of these curves is the appearance of multiple plateaus on the 
otherwise monotonically decreasing plots. In addition, the plateaus appear to be spaced quite evenly. Closer 
analysis of these plots indicates that the plateaus occur wherever local link-stages are used in the network. 
In partk:ular, the packet loss probability does not seem to be significantly decreased by the inclusion of many 
of the local link-stages. This should be expected, because local link-stages are only useful if a packet is de- 
flected by a full set of output chutes in its current output row. The local link-stage can route the packet to a 
different output row that is still connected to its desired output packet module. Having a full set of output chutes 
in a row is rare, because most practical networks will have relatively large values of m to maintain a low grow- 
able packet switch limit (FIG. 23), so they will also have a fairly large numberof output chutes (2m/n) connected 
to each node. In addition, it is even more unlikely that the local link-stages will be used near the input end of 
the network, because the propagating packets have not yet passed through enough stages to fill the output 
chutes. 

Thus the local link-stages can oftentimes be eliminated from the Canyan network design. The resulting 
network topology typically requires fewer node-stages for a particular packet loss probability, so these net- 
works are called Compressed-Canyan networks. (Note: If a Canyan network is designed with a small number 
of output chutes per node, or If extremely low blocking probabilities are required, or if "hot spots" (popular output 
packet modules) are expected, then local link-stages can typically be added near the middle and output end 
of the network to correct for the problems associated with full output chutes). 

Unfortunately, all of the local link-stages from a Canyan network cannot be removed to create a useful 
Compressed-Canyan network. The link-stages to be removed are carefully selected. For example, if a partic- 
ular Canyan network is designed with a small number of output chutes per node, or if extremely low blocking 
probabilities are required, or if "hot spots" (popular output packet modules) are expected, then local link-stages 
can typically be added near the mkldle and output end of the network to correct for the problems associated 
with full output chutes, in addition, it may be necessary to customize the type and number of consecutive local 
link-stages that are used in each of these correction regk>ns. 

As an example, the packet loss probability curves for three Compressed-Canyan networks (L=N=64, 
L=N=256, and L=N=1024) are illustrated in FIG. 25. Each of the networks has the same network parameters 
(m=32, n=8, and pl=1.0). and each of the networks has all of its local link-stages removed. Unfortunately, the 
problems associated with full output chutes become apparent In each of these plots, because the plots level 
out at packet loss probabilities much higher than those predicted by the growable packet switch limit To correct 
for the full output chute problems in these three networks, local link-stages are added to the networks in stra- 
tegic locations. The following iterative mapping rules (Eqs. 1 5-1 8) were used to determine where and how local 
link-stages should be added to improve the networks of FIG. 25. 
For link-stage 1=0, the connection variable V(0) is given by: 

. V(0) = log2(N).1. [15] 
For link-stage 1, 1^i<Iog2(N)log2(N/n)/2, the connection variable V(i) is given by: 

V(l) = V(i - 1) . 1. If V(i - 1)^log2(n) + 1 [16] 
V(i) = log2(N) - 1. If V(i - 1) = log2(n) 
For link-stage i=log2(N)log2(N/n)/2. the connection variable V(i) is given by: 

V(i) = log2(n)-1. [17] 
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For link-stage I. i > log2(N)log2(N/n)y2, the connection variable V(i) is given by: 

V(l) = V(l - 1) - 1. if V(i - 1)^log2(n) + 1 [18] 
V(i) = V(l - log2(N/n) - 1), If (V(i - 1) = log2{n)] AND [V(i - log2(N/n) - 1)>1] 
V(i) = log2(n) - 1 , if [V(l - 1) = log2(n)] AND [V(l - log2(N/n) - 1) = 1] 
if these rules are applied to the networks In FIG. 25, new Compressed-Canyan networks with strategic local 
link-stages are created. The loss probability curves for the three new Compressed-Canyan networks are illu- 
strated in FIG. 26. Comparisons between FIG. 21 and FIG, 26 indicate that the use of Compressed-Canyan 
networks result In substantial savings in system hardware costs. 

The Iterative mapping rules described above were devised using trialand-error techniques. 

Claims 

1. A packet switching arrangement having L input ports. M intermediate ports and N output ports, where U 
M and N are positive integere greater than two, said arrangement comprising 

a distribution network for switching packets from said L input ports to said M intermediate ports 
without storing full received packets, and 

N/n output packet switching modules, where n and N/n are positive integere with N>n>1, each of 
said modules comprising 

means for switching packets from a distinct group of m of said Intermediate ports to a distinct group 
of n of said output ports, where m is a positive integer with m>n. and 

means for storing full received packets, 

said distribution network comprising 

S stages of nodes, with each node of at least the first (S-1) of said node stages comprising an X 
X Y switching means, where S, X and Y are positive Integere with S>3, X>1 and Y>1. and 

(S-1) link stages interconnecting successive ones of said S node stages, with each link stage com- 
prising chute links and switch links, 

wherein t he X x Y switching means of each node of the Interior (S-2) of said S node stages receives 
packets on X switch links and selectively transmits packets on Y switch links. 

wherein said each node of said interior (S-2) node stages further comprises 

a plurality of chute connection means each connected to only one chute link of the preceding link 
stage and to only one chute link of the following link stage, for transmitting a packet, being received from 
said preceding stage chute link, onto said following stage chute link, and 

means for transferring a packet, being received on a switch link by the X x Y switching means of 
said each node of said Interior (S-2) node stages, to any selected one of said plurality of chute connection 
means for transmission on the following stage chute link. 

2. An arrangement in accordance with daim 1 wherein the number of said plurality of chute connection 
means of said each node of said interior (S-2) node stages Is XnVn, where m/n Is a positive integer. 

3. An arrangement in accordance with claim 1 wherein all of said plurality of chute connection means of said 
each node of said interior (S-2) node stages are connected via chute links to only one node of the following 
node stage. 

4. An arrangement In accordance with daim 1 wherein said selected chute connection means is selected 
based on the absence of a packet on the preceding stage chute link. 

5. An arrangement in accordance with daim 1 wherein said transferring means Is responsh^e to destination 
information induded in said packet being received on a switch link. 

6. An arrangement In accordance with claim 1 wherein said transferring means Is responsh/e for transferring 
said packet being received on a switch link unless packets are present on all of the preceding stage chute 
links connected to said plurality of chute connection means of said each node of said interior (S-2) node 
stages. 

7. An arrangement in accordance with daim 1 wherein 

the X X Y switching means of each node of the first of said S node stages receives packets from 
X of said input ports and selectively transmits packets on Y switch links, and 

wherein said each node of said first of said S node stages further comprises 
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a plurality of chute connection means each connected to oniy one chute link of the following link 
stage, for transmitting a packet onto the following stage chute link, and 

means for transferring a packet, being received from one of said X input ports by said X x Y switch- 
ing means of said each node of said first of said S node stages, to any selected one of said plurality of 
5 chute connection means of said each node of said first of said S node stages for transmission on the fol- 

lowing stage chute link, and 

wherein each node of the last of said S node stages comprises 

means for receiving packets on X switch links, 

a plurality of chute connection means each connected to only one chute link of the preceding link 
10 stage and to only one of said M intermediate ports, for transmitting a packet, being received from the pre- 

ceding stage chute link, to said one intermediate port, and 

means for transferring a packet, being received on a switch link by said receiving means, to any 
selected one of said plurality of chute connection means of said each node of said last of saki S node 
stages for transmission to said one intermediate port. 

IS 

8. An arrangement In accordance with claim 7 wherein said receiving means discards a packet being re- 
ceived on a switch link by said receiving means but not being transferred by said transferring means of 
said each node of the last of said S node stages. 

2Q 9. An arrangement in accordance with claim 7 where no packets are discarded by any nodes of the first (S- 
1) of said S node stages. 

10. An arrangement In accordance with claim 1 wherein said distribution network comprises a plurality of par- 
tittons each associated with a distinct one of said output packet switching modules and each comprising 
all of the chute links of said network leading to the associated output packet switching module and 
all of the nodes of said network interconnected by said all of the chute links. 
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11. An arrangement In accordance with claim 10 wherein said each partition comprises 
a plurality of nodes in each of said S node stages. 

^ 12. An arrangement in accordance with daim 10 wherein said each partition comprises 
n/X nodes in each of said S node stages. 

13. An arrangement in accordance with daim 10 wherein said (S-1) link stages comprise global link stages 
having inter-partltton switch links and local link stages having no inter-partition switch links and wherein 

35 the nodes of the first (S-1) of said S node stages comprise global nodes preceding global link stages and 

local nodes preceding local link stages, wherein global nodes route packets in accordance with a first rout- 
ing method, and local nodes route packets in accordance with a second routing method. 

14. An arrangement in accordance with daim 1 wherein each of said chute connectton means comprises a 
40 FET-SEEO circuit. 
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FIG, 9 



IS PACKET A 
^IN A NODE CONNECTED 
VTO ITS DESIRED OUTPUT, 
^PACKET MODULE? 
iYES~" 

IS THERE 
AN AVAIUBLE 



(FIG. 10) 



\ OUTPUT CHUTE / 
\ IN THIS NODE? / 




YES 


ROUTE PACKET A 
TO THE AVAIUBLE 
OUTPUT CHUTE 








ROUTE PACKET A 
TO OUTPUT UNK C 



IS PACKET B 
AN ACTIVE PACKET? 



I YES 



\no 



IS PACKET B 
AN ACTIVE PACKET? 



\no 



lYES 

<D0ES PACKET B'a \ no 
routing BIT >^ 
R = 0 ? / 

Iyes 



ROUTE PACKET B 
TO OUTPUT UNK C 
k ZEROES TO UNK D 



ROUTE PACKET B 
TO OUTPUT UNK 0 



ROUTE ZEROES TO 
OUTPUT UNKS C & 0 



ROUTE PACKET B 
TO OUTPUT UNK D 
ft ZEROES TO UNK C 



ROUTE ZEROES 
TO OUTPUT UNK D 



37 



EP0 590 865 A2 



FIG, 10 
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FIG. 11 
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FIG. 12 
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FIG, 13 
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FIG. 14 
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FIG. 15 
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